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ABSTRACT 

The probability sample was a two-stage stratified 
sample. The first-stage units were schools and the second-stage units 
were students. Because certain subpopulations were overs ampled, the 
data had to be weighted for analysis purposes* The analyses concerned 
contrasting North Carolina and its various subpopulations with the 
nation on standardized aptitude and achievement tests. Constrasts 
between various subpopulations defined by geographical region, type 
of community, and socioeconomic status within North Carolina, were 
also made on cognitive and non-cognitive output measures. In 
addition, the relationship of various school process variables to 
achievement measures was investigated. (Author) 
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Analysis of Data from the North Carolina Statewide 
Assessment of Educational Progress 

Previous papers in this series described the instrumentation, data 
collection procedures, and the sample design for the North Carolina 
Educational Assessment. This paper briefly describes the analysis of 
the assessment data. 

The analyses conducted on the North Carolina Assessment Data were 
primarily descriptive. They were supplemented, however, by a limited 
number of weighted and unweighted regression analyses. One purpose of 
these descriptive a nalyses was to estimate the proportion of North 
Carolina 6th grade students who belonged to various important subpopulations 
of North Carolina such as Title I Reading program participants. Title I 
Language Arts program participants. Title I Math program participants, 
proportion of 6th graders with previous Title I experience, 6th graders 
who had attended kindergarten, and 6th graders with remedial needs in 
Reading, Language Arts, and Mathematics. The remedial needs were determined 
from both teacher^s ratings and the Iowa Test of Basic Skills (ITBS). 
The proportion of children with remedial needs in Reading, Language Arts, 
and Mathematics that were in appropriate Title I and other remedial 
programs was also estimated. These last estimates are important since 
they not only reflect remedial needs but In addition give some indication 
of the extent to which these needs are being satisfied. 

Another purpose of the descriptive analyses was to obtain status 
information on school characteristics such as adequacy of physical facilities, 
adequacy of personnel resources, and types of educational programs and 
practices (for example, ungraded classrooms). 

The most important descriptive analyses were those aimed at portraying 
the ability and achievement status of various subgroups of children. There 
was interest in estimating the ability and achievement level of children 
from three geographical regions of North Carolina (Coastal Plains, Piedmont, 
and Mountain), three types of communities (Large City, Large City Fringe/ 
Medium City, and Rural/Small Town), and Title I versus non-Title I. To 
insure adequate sample sizes for each of these subpopulations, some of the 
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subpopulations were oversampled . The distribution of the sample also allowed 
us to estimate ability and achievement levels by ethnicity, sex, and parentis 
education level. The weighted means were calculated for various subpopulations 
based on the above variables for Academic Ability as measured by the Lorge- 
Thorndike, ITBS Reading, Language Arts, and Mathematics subscores, the 
Cognitive Test of Vocational Maturity which measured Career Awareness, a 
Mathematics Test constructed by the North Carolina Department of Public 
Instruction, and a student survey instrument measuring student attitudes 
and perceptions towards teachers and schools. This last instrument was also 
developed by the North Carolina Department of Public Instruction. 

The standard errors of the means for f'M * f i ll rnhp^F^ ^ I ''^Tf^^ '^ 

were estimated by the appropriate formula for two stage stratified sanpling. 
In addition to estimating means for these subpopulations of interest, various 
mean differences of interest and their standard errors were also estinated. 
On all tests, the following contrasts were made: between regions, between 
types of communities, between ethnicities, between sexes, between Title I, and 
other remedial and non-remedial pro'grams, and between parent education levels. 

The weighted state estimate of both the mean and the median for each 
of the ITBS Math, Reading, and Language Arts subscores was obtained and 
contrasted with the medians for the Nation and the Southeast. Norms were 
developed for North Carolina, the three geographical regions, and the three 
types of communities. These norms were developed to satisfy a desire on 
the part of North Carolina educators. The possibility of developing school 
norms based upon school means is also being explored. School norms might 
be more relevant than individual norm& since they allow school officials, 
to examine the performance of their school relative to other schools. 
Individual norms are not appropriate for this purpose. 

Weighted and unweighted regression analyses were also run contrasting 
regions and TOCs while adjusting for background factors such as ethnicity, 
and parentis education level. Weighted and unweighted regression analyses 
were also run contrasting Title I, non-Title I remedial, and non-remedial 
6th graders for all achievement test scores while adjusting for student 
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background factors such as ethnicity and parentis education level. The 
weighted regression analyses in/olved estimating the population covariance 
matrix by calculating a weighted covariance matrix based upon the sample 
weights. It turned out that the results from both the unweighted and 
weighted regression analyses were highly similar to each other. 

The above analyses were concerned with describing the status of sub- 
populations of children where the subpopulations were defined by background 
variables of the students such as parental education level, ethnicity, and 
geographical location. They are useful for looking at the educational needs 
of children from different backgrounds and circumstances. Background factors 
were more important than geographical and type of community factors in 
predicting achievement. But nevertheless since background attributes are 
confounded with geographical region and type of community the problem of 
low achievement is more fevident in some types of communities and regions 
than others. Like most studies, the assessment results verified the strong 
relationship of child background factors with achievement. 

In addition to estimating the proportion of 6th graders in various 
subpopulations, and estimating the achievement levels for these various 
subpopulations, the proportion of schools that fell into various categories 
was estimated. Some of the more interesting subpopulations were defined by 
program characteristics. In addition, the proportions of principals holding 
various perceptions concerning the adequacy of school facilities and teaching 
staff were estimated. Also, the proportions of children holding various 
attitudes and perceptions of teachers, classroom and school attributes were 
estimated. 

Due to time and cost constraints, the achievement levels for various 
types of schools were not estimated. Nor where there any studies involving the 
construction of "regression like" models that considered the joint influence of 
both school and student background characteristics on achievement. 

The data file now available is a rich one indeed containing ability, 
achievement, actitudinal, socioeconomic, and other background factors 
on approximately 11,000 children. In addition, each child ^s record 
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contains information concerning the child •s educational environment (i.e., 
school and program characteristics). Each child's record also contains 
a wealth of information concerning remedial program participation. Most 
of the analyses that we have described could be considered preliminary 
in nature. 

There is a definite need for more extensive analyses on this data set. 
Some of the further analyses should involve considering the joint influence 
of child background and school factors on the achievement of North Carolina 
6th graders by examining the estimated parameters for child and school 
factorc for vnrJ . ous gtatisi eal models of the education process; examining 
in greater detail the characteristics of Title I and other remedial programs 
and their relationship with academic achievement; developing realistic 
path analysis or other causal models of the educational system for North 
Carolina; and developing reliable and valid attitudinal measures and other 
non-cognitive measures from the many items of the student survey instrument. 
There is a tremendous need for good measures of this sort. Good measures 
such as these vrculd certainly play an important role in any statistical 
modeling of the education process for North Carolina 6th grade students. 

When analyses like these have been completed, then the next step is 
to design a follow-up assessment strategy to measure change and those 
factors related to change. The present data establishes a baseline for 
future assessments. 

We will now present some selected results. In general. North Carolina 
scored substantially below the Nation on all Math, Reading, and Language 
Arts subtests and significantly but not substantially below the norms for the 
Southeast. However, the following facts must be considered in interpreting 
these findings. First, although the distribution of scores for North 
Carolina were based upon a carefully designed probability sample, the ITBS 
norms for the Nation and the Southeast were not developed from a 
probability sample and hence could be somewhat biased. Second, North Carolina 
ranks low on per capita income, is highly rural, and has a high percentage 
of Blacks compared to the Nation. All of these factors are substantially 
related to achievement. 

The means on all of the Math, Reading, and Language Arts subtests for 
the ITB.; were contrasted for the three regions within North Carolina. In 



ERLC 



4 



general, the means for the Coastal Plains 6th grade students were 
significantly below the corresponding means of the other two regions. On 
the other hand, the Mountain Region 6th grade students tended to score 
higher than the other two regions. These results could be due to a number 
of reasons including the facts that Blacks had a relatively high density 
in the Coastal Plains Region and that this Region is extremely rural with 
a low SES level. In fact, some further analytical work indicated that when 
adjusted for factors like parent's education and ethnicity, these regional 
differences became smaller and for some subtests of the ITBS disappeared. 

The same types of descriptive and analytical procedur^es.ji7ere_used to 
examine the differences between the three types of communities. There 
was a general tendency for children residing in rural/small town areas to 
have the lowest achievement level on the ITBS. Regression adjustments on 
the basis of child background factors such as ethnicity and parent's 
occupation further reduced these differences. 

The results for the Cognitive Vocational Maturity Test and the Student 
Survey Questionnaire will not be discussed because less is known of their 
psychometric and measurement properties. 

As expected, females scored significantly but not substantially higher 
than the males on all ITBS measures. Parent's education (defined as the 
highest education level of either parent) was also substantially related 
to ITBS achievement. The non-White group of North Carolina 6th grade 
children scored substantially below the White group in all achievement 
areas. The non-White group was predominantly Black and was roughly 
a year behind in achievement. 

In addition to providing information on the relative achievement for 
various subgroups of North Carolina children, the results indicated that only 
a small percentage of the 6th graders who scored low on the ITBS were in 
special remedial programs such as Title I. 

Of the three basic instructional programs, reading was the most heavily 
enrolled Title I program. Language Arts was second, and Mathematics was 
third. The characteristics of Title I and non-Title I were contrasted. 



The achievement level of Title I children in the basic skillr. was 
substantially lower than the non-Title I participants. We wculd, of 
course, expect this since Title I children are more likely to be non-Wiite, 
of lower ability, and com^. from a less advantageous home situation relative 
to non-Title I participauts. They have been selected for Title I 
participation on the basis of low achievement and achievement is highly 
related to these factors • 

Children were also classified in regard to type of remedial reading 
participation: Title I Reading (13 percent of Staters 6th graders), non- 
Title I Remedial Reading (7 percent), and no Remedial Reading (81 percent). 
The children in non-Title ^ Remedial Reading programs did significantly and 
substantially worse than Title I children. The same trends were found for 
Mathematics and Language Skills, The ITBS means for these three groups 
were adjusted for the effects of region, size of community, ethnicity, and 
aptitude. The adjusted means were not substantially different from one 
another. 



